Skip to content

fix(net): reuse DPDK handles after pending I/O retires - #32

Merged
thweetkomputer merged 2 commits into
mainfrom
fix/dpdk-handle-reuse
Nov 7, 2026
Merged

thweetkomputer merged 2 commits into
mainfrom
fix/dpdk-handle-reuse

Conversation

@thweetkomputer

@thweetkomputer thweetkomputer commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

DPDK socket handles currently increase monotonically per worker, so a long-running worker eventually returns EMFILE even when almost all connections have closed. This change recycles closed handle slots once all pending operations and completion callbacks for the old socket have retired.

Implementation

  • Replace the monotonic handle allocator with a worker-local slot table and free list.
  • Pin the socket identity through pending send, sendmsg, connect, accept, receive, disconnect, cancellation, and completion delivery. Closing invalidates the transport immediately; pending operations cannot act on a replacement socket.
  • Preserve worker-encoded handles and public socket APIs without a generation counter or cross-worker locking.
  • Retain reusable slot metadata at peak live/draining socket capacity; closed slots retain no transport or payload buffers.
  • Update the socket lifetime comments and networking architecture document.

Validation

Run from the Bycorf checkout on ARM64 with Clang 18:

ctest --test-dir build-bypass -R 'DpdkSocketTableTest|DpdkHandleReuseDeathTest' --output-on-failure
ctest --test-dir build-bypass -R 'Connect|ConnectionStorage|Http|Tcp|Uring|DpdkAdaptive|FreeBsdLocal' --output-on-failure
ctest --test-dir build-asan -R 'DpdkSocketTableTest|Connect|ConnectionStorage|Http|Tcp|Uring' --output-on-failure
  • 18 new tests pass: 6 slot lifetime tests and 12 DPDK backend scenarios covering churn, pending operations, cancellation, callbacks, and shutdown.
  • 34 existing networking regressions pass.
  • 31 slot and kernel runtime tests pass under ASAN/UBSAN. The sanitizer run does not cover the DPDK backend.
  • DPDK scenarios use the real EAL/ring PMD and BSD socket lifecycle with controlled send/connect readiness. Physical NIC behavior was not tested.

Integration

Lavik only needs the Bycorf submodule update; its networking call sites, including WriteAll and WriteAllV, remain unchanged.

Lavik integration: eloqdata/lavik#455. Merge this PR first; refresh Lavik's gitlink to the resulting main-branch commit if its SHA changes.

Replace cumulative worker-local handle allocation with reusable slots. Pending operations and completion delivery pin each slot until the old socket can no longer be observed, including cancellation and shutdown. Keep transport close immediate and preserve the public socket APIs.

Validate with 6 slot lifetime tests, 12 DPDK ring backend scenarios, and 34 existing networking regressions. Additionally pass 31 slot and kernel runtime tests under ASAN/UBSAN. Physical NIC behavior was not exercised.
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e3f275bc-5d0c-410f-8459-0e0623d53bc0
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@thweetkomputer
thweetkomputer merged commit ad14f25 into main Nov 7, 2026
10 checks passed
@thweetkomputer
thweetkomputer deleted the fix/dpdk-handle-reuse branch November 7, 2026 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant